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Abstract. We derive quantitative results regarding sets of n-bit strings that have different 
dependency or independency properties. Let C{x) be the Kolmogorov complexity of the string 
X. A string y has a dependency with a string x if C{y) ~ C{y \ x) > a. A set of strings 
{xi, . . . ,xt} is pairwise a-independent if for all i 7^ j, C(xi) — C{xi \ Xj) < q. A tuple of 
strings (x\, . . . , xt) is mutually a-independent if C(2;^(i) . . . a;^(t)) > C{xi) + . . . + C{xt) — a, 
for every permutation tt of [t]. We show that: 

— For every n-bit string x with complexity C{x) > a + 71ogn, the set of n-bit strings that 
' have a dependency with x has size at least (l/poly(n))2"~". In case a is computable from 

OA , n and C{x) > a + 121ogn, the size of same set is at least (1/C)2"~" — poly(n)2°, for 

' some positive constant C. 

— There exists a set of n-bit strings A of size poly (n) 2" such that any n-bit string has 
a-dependency with some string in A. 



1—5 



— If the set of n-bit strings {xi,...,xt} is pairwise a-independent, then t < poly(n)2°'. 
This bound is tight within a poly(n) factor, because, for every n, there exists a set of 
, n-bit strings {xi, . . . ,xt} that is pairwise a-dependent with t — (l/poly(n)) ■ 2" (for all 

C.) ' a > 51ogn). 
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If the tuple of n-bit strings {xi, . . . ,Xt) is mutually a-independent, then t < poly(n)2° 
(for all Q > 71ogn + 6). 



^ ! 1 Introduction 

in 

, A fact common to many mathematical settings is that in a sufficiently large set some 

relationship emerges among its elements. Generically, these are called Ramsey-type results. 
' We list just a few examples: any n + 1 vectors in an n-dimensional vector space must be 

dependent; for every k and sufficiently large n, any subset of [n] of constant density must 
have k elements in arithmetic progression; any set of 5 points in the plane must contain 4 
points that form a convex polygon. All these results show that in a sufficiently large set, 
some attribute of one element is determined by the other elements. 

We present in this paper a manifestation of this phenomenon in the very general frame- 
\ work of algorithmic information theory. We show that in a sufficiently large set some form of 

algorithmical dependency among its elements must exist. Informally speaking, poly(n) • 2" 
binary strings of length n must share at least a bits of information. For one interpretation 
of "share" , we also show that this bound is tight within a poly(n) factor. 

Central to our investigation are the notions of information in a string and the derived 
notion of dependency between strings. The information in a string x is captured by its 
Kolmogorov complexity C{x). A string y has a-dependency with string x if C{y)—C{y \ x) > 
a. The expression C(y) — C(y | x), denoted usually more concisely as I{x : y), represents 
the quantity of information in x about y and is a key concept in information theory. It 
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is known that I{x : y) = I{y : x) it O(logn) (Symmetry of Information Theorem [20]), 
where n is the length of the longer between the strings x and y, and therefore I{x : y) is 
also called the mutual information of x and y. For any n-bit string x and positive integer 
a, we are interested in estimating the size of the set Ax^a of n-bit strings y such that 
C{y) — C{y \ x) > a. One can see by a standard counting argument that |^a;,a| < 2"'""^'^ 
for some constant c. Regarding a lower bound for lAj^^aj, it is easy to see that if C{x) ■< a, 
then Ax^a is empty (intuitively, in order for x to have a bits of information about y, it 
needs to have a bits of information to start with, regardless of y). The lower bound that 
we establish holds for any string having Kolmogorov complexity ^ a0 For such strings 
X, we show that \Ax^a\ > (l/poly(n))2"'~". A related set is Bx^a consisting of the n-bit 
strings y with the property C {y \ n) — C {y \ x) > a. This is the set of n-bit strings about 
which X has a bits of information besides the length. Note that Bx^a ^ The same 

observations regarding an upper bound for |-Bx,a| and the emptiness of Bx^a in case C{x) ^ a 
remain valid. For x with C{x) ^ a and a computable from n, we show the lower bound 
> (1/C) • 2"~" — poly(n) • 2", for some positive constant C . 

We turn to the Ramsey-type results announced above. A set of n-bit strings {xi, . . . , xt} 
is pairwise a-independent if for all i ^ j, C{xi) — C{xi \ Xj) < a. Intuitively, this means 
that any two strings in the set have in common at most a bits of information. For the 
notion of mutual independence we propose the following definition (but other variants are 
conceivable). The tuple of n-bit strings (xi, . . . ,xt) G ({0, 1}")* is mutually a-independent 
if C(x7r(i) . . . x^(i)) > C{xi) -|- . . . -|- C{xt) — a, for every permutation vr of [t]. Intuitively 
this means that xi, . . . ,xt share at most a bits of information. We show that if {xi, . . . , xt} 
is pairwise a-independent or if (xi, . . . ,xt) is mutually a-independent then t < poly(n)2°. 
The bound in the pairwise independent case is tight within a polynomial factor. 

We also show that there exists a set B of size poly (n) 2" that "a-covers" the entire set 
of n-bit strings, in the sense that for each n-bit string y there exists a string x in 5 that 
has a bits of information about y (i.e., y is in Ax^a)- 

The main technical novelty of this paper is the technique used to lower bound the 
size of Bx^a = {y S {0,1}" | C{y \ n) — C{y \ x) > a}, which should be contrasted 
with a known and simple approach. This "normal" and simple approach is best illustrated 
when X is random. In this case, the prefix x(l : a) of x of length a is also random and, 
therefore, if we take z to be an (n — a) long string that is random conditioned by x(l : a), 
then C{zx{l : a)) = n — O(logn), C{zx{l : a) | x(l : a)) = n — a — O(logn), and 
thus, 2:x(l : a) € -Bx,a+o(iogn)- There are approximately 2"~" strings z as above, and 
this leads to a lower bound of 2"~" for |-Ba:,a+o(iogn)l) which implies a lower bound of 
(l/poly(n))2"~" for This method is so basic and natural that it looks hard to beat. 

However, using properties of Kolmogorov complexity extractors, we derive a better lower 
bound for iS^^al that does not have the slack of l/poly(n), in case a is computable from n 
(even if a is not computable from n, the new method gives a tighter estimation than the 
above "normal" method). A Kolmogorov complexity extractor is a function that starting 
with several strings that have Kolmogorov complexity relatively small compared to their 
lengths, computes a string that has Kolmogorov complexity almost close to its length. A 
related notion, namely multi-source randomness extractors, has been studied extensively 

^ We use notation poly(n) for n'^'^' and ~, ^ and >: to denote that the respective equality or inequahty 
holds with an error of at most O(logn). 
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in computational complexity (see |3|l|2|12|llj ). Hitchcock, Pavan and Vinodchandran [8] 
have shown that Kolmogorov complexity extractors are equivalent to a type of functions 
that are close to being multisource randomness extractors. Fortnow, Hitchcock, Pavan, 
Vinodchandran and Wang [7] have constructed a polynomial-time Kolmogorov complexity 
extractor based on the multi-source randomness constractor of Barak, Impagliazzo and 
Wigderson [1]. The author has constructed Kolmogorov complexity extractors for other 
settings, such as extracting from infinite binary sequences [18|16j or from binary strings that 
have a bounded degree of dependence [16|19|17] . The latter type of Kolmogorov complexity 
extractors is relevant for this paper. Here we modify slightly an extractor E from [17j , 
which, on inputs two n-bit strings x and y that have Kolmogorov complexity at least s and 
dependency at most a, constructs an m-bit string z with m ^ s and Kolmogorov complexity 
equal to m — a — 0(1) even conditioned by any one of the input strings. Let us call a pair 
of strings x and y with the above properties as good- for- extraction. We fix x G {0, 1}" with 
C{x) > s. Let z be the most popular image of the function E restricted to {x} x {0, 1}". 
Because it is distinguishable from all other strings, given x, z can be described with only 
0(1) bits (we only need a description of the function E and of the input length). Choosing 
m just slightly larger than a we arrange that C{z\x)<m — a — 0(1) . This implies that 
all the preimages of z under E restricted as above are bad- for- extraction. Since the size 
of E^^{z) n {{x} X {0,1}") is at least 2""'", we see that at least 2"^™" pairs {x,y) are 
bad-for-extraction. A pair of strings (x, y) is bad-for-extraction if either y has Kolmogorov 
complexity below s (and it is easy to find an upper bound on the number of such strings), 
or if ?/ € Bx^a- This allows us to find the lower bound for the size of B^^a- 

2 Preliminaries 

We work over the binary alphabet {0, 1}; N is the set of natural numbers. A string x is an 
element of {0, 1}*; |x| denotes its length; {0, 1}" denotes the set of strings of length n; \A\ 
denotes the cardinality of a finite set A; for n € N, [n] denotes the set {1,2,..., n}. We 
recall the basics of (plain) Kolmogorov complexity (for an extensive coverage, the reader 
should consult one of the monographs by Calude 0, Li and Vitanyi [10], or Downey and 
Hirschfeldt [6]; for a good and concise introduction, see Shen's lecture notes [13])- Let M 
be a standard Turing machine. For any string x, define the (plain) Kolmogorov complexity 
of X with respect to M, as 



There is a universal Turing machine U such that for every machine M there is a constant 
c such that for all x, 



We fix such a universal machine U and dropping the subscript, we let C{x) denote the 
Kolmogorov complexity of x with respect to U. We also use the concept of conditional 
Kolmogorov complexity. Here the underlying machine is a Turing machine that in addition 
to the read/work tape which in the initial state contains the input p, has a second tape 
containing initially a string y, which is called the conditioning information. Given such a 



Cm{x) 



min{|p| I M{p) = x}. 



Cu{x) < Cm{x)+c. 



(1) 
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machine M, we define the Kolmogorov complexity of x conditioned by y with respect to M 
as 

Cm{x \ y) = min{|p| | M{p,y) = x}. 

Similarly to the above, there exist universal machines of this type and they satisfy the 
relation similar to Equation [H but for conditional complexity. We fix such a universal 
machine U, and dropping the subscript U, we let C{x \ y) denote the Kolmogorov complexity 
of X conditioned by y with respect to U . 

There exists a constant cu such that for all strings x, C{x) < \x\ + cij. Strings 
xi,X2, ■ ■ ■ ,Xk can be encoded in a self-delimiting way (i.e., an encoding from which each 
string can be retrieved) using |xi| + |x2| + • • • + + 2 log |a;2| + . . . + 2 log \xk\ + 0{k) 
bits. For example, xi and X2 can be encoded as (6m(|x2|)01xiX2, where hin{n) is the 
binary encoding of the natural number n and, for a string u = ui . . . Um, u is the string 
uiui . . . UmUm (i-G., the string u with its bits doubled). 

Given a string x and its Kolmogorov complexity C(x), one can effectively enumerate all 
descriptions y of x of length C(x), i.e., the set {y G {0, | U{y) = x}. We denote x* the 
first string in this enumeration. Note that C(x)- 0(1) < C{x*) < |x*|-|-0(l) = C{x)+0{1). 

The Symmetry of Information Theorem [20] states that for any two strings x and y, 

(a) C{xy) < C{y) + C{x\y) + 2 log C{y) + 0(1). 

(b) C{xy) > C{x) + C{y \ x) - 2 log C{xy) - AloglogC (xy) - 0{1). 

(c) If l^l = \y\ = n, C{y) — C{y \ x) > C{x) — C{x | y) — 51ogn 

Since the theorem is usually stated in a slightly different form and since we use the constants 
specified above, we present in the appendix the proof (which follows the standard method). 

As discussed in the Introduction, our main focus is on sets of strings having certain de- 
pendency or independency properties. For convenience, we restate here the main definitions. 

Definition 1. The string y has a-dependency (where a G with the string x if C{y) — 
C{y \ x) > a or if X coincides with y. 

We have included the case "x coincides with y" to make a string dependent with itself even 
in case it has low Kolmogorov complexity. 

Definition 2. The strings xi, . . . ,Xj are pairwise a-independent if for all i ^ j, C{xi) — 
C{xi I Xj) < a. 

Definition 3. The tuple of strings {xi, . . . ,xt) is mutually a-independent (where a €f^) if 
C'(x7r(i)X^(2) • • • a^7r(t)) ^ C (xi) + C {X2) -|- . . . -|- C{xt) — Ci, for every permutation vr of [t\. 

3 Strings dependent with a given string 

Given a string x G {0, 1}", and a € N, how many strings have dependency with x at least 
a! That is we are interested in estimating the size of the set 

= {y e {0, 1}" I C{y) - C{y I x) > a}. 
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This is the set of strings about which, roughly speaking, x has at least a bits of information. 
A related set is 

Sx,a = {y e {0, 1}" I Ciy I n) - C{y \ x) > a}, 

consisting of the n-bit strings about which x provides a bits of information besides the 
length n. Clearly, Bx,a C Ax,a, and thus an upper bound for \Ax,a\ also holds for |Sa;,a|, 
and a lower bound for |-Bx,a| also holds for 

We show that for some polynomial p and for some constant C, for all x and a except 
some special values, 

(lMn))-2"-"< <C2"-'^, 
and, in case a(n) is computable from n, 

(1/C) ■ 2"-" -p(n)2" < \Bx,a\ < C2"-", 

The upper bounds for the sizes of A^^a and B^^a can be readily derived. Observe that the 
set Ax^a is included in {y G {0, l}" | C{y | x) < n — a + c} for some constant c, and therefore 

|^a;,a| < C • 2*^ 

for C = 2". 

We move to finding a lower bound for the size of Ax^a- A first observation is that for 
Ax,Q. to be non-empty, it is needed that C{x) >_ a. Indeed, it is immediate to observe that 
for any strings x and y of length n, 

Ciy) < C{x) + C{y \ x) + 2logC{x) + 0(1) < C{x) + C{y | x) + 21ogn + 0(1), 

and thus, if C{y) — C{y \ x) > a, then C{x) > a — 21ogn — 0(1). Intuitively, if the 
information in x is close to a, not too many strings can be a-dependent with it. 

We provide a lower bound for lA^i^a], for every string x with C{x) > a + Tlogn. The proof 
uses the basic " normal" approach presented in the Introduction. To simplify the discussion, 
suppose C{x) = a. Then if we take a string z of length n — a that is random conditioned by 
X*, it holds that C{x*z) f« n and C{x*z \ x*) n-a. Thus, C{x*z)-C{x*z \ x*) >: a. Note 
that there are approximately 2"^" such strings x*z. Since x* can be obtained from x and 
C{x), we can replace x* by x in the conditioning at a small price. We obtain approximately 
2n-a strings in Ax,a- 

Theorem 1. For every natural number n, for every natural number a and for every x G 
{0, 1}" such that C{x) >a + 7\ogn, 

\A I > o"-Q 

provided n is large enough. 

Proof. Let k = C{x) and let /3 = a + 71ogn. Let x* be the smallest description of x as 
described in the Preliminaries. Let x*^ be the prefix of x* of length p. Since x* is described 
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by and by its suffix of length k-f3, C{x*) < C(x^) + (A; - /3) + 2 log C{x*^) + 0(1) and, 
thus 

Cix;) > C{x*) -ik-/3) - 21ogC(x*) - 0(1) 

>{k- 0(1)) -{k-(3)-2 log C{x;) - 0(1) 
>/3-21og/3-0(l). 

The set B = {z e {0, l}"-/^ | C(z | x^) > n - /3 - 1} has size at least (1/2) • 2"-/^ (using 
a standard counting argument). Consider a string y G {0,1}"' of the from y = x*^z with 
z e B. There are at least (1/2) • 2"~^ such strings. 
By symmetry of information, 

C{y) = C{x}z) > Cix}) + C(z I x}) - (2 log n + 4 log log n + 0(1)) 

> (/3 - 2 log /3) + (n - /3 - 1) - (2 log n + 4 log log n + 0(1)) 

> n — (41ogn + 4 log log n + 0(1)) > n — 51ogn. 

On the other hand, C{y \ ) = Cix^z \ x*^) < C{z) + 0(1) < (n - /3) + 0(1). Note that 

C{y I x) < C{y I x*p) + 21ogn + 41oglogn + 0(1), 
because one can effectively construct x*^ from x, k and (3. Therefore, 
C{y \x) < {n- /3) + 21ogn + 41oglogn + 0(1), 

and thus 

C{y)-C{y I x) > /3 - (61ogn + 81oglogn + 0(l)) >/3-71ogn. 

So, y G Axfi-7\ogn = Ax,a- Since this holds for all the strings y mentioned above, it follows 
that 1^^. „|' > (1/2)2"-^ = (l/(2n7)) • 2""°. | 

The lower bound for is obtained using a technique based on Kolmogorov com- 

plexity extractors, as explained in the Introduction. We use the following theorem which 
can be obtained by a simple modification of a result from [T?]. 

Theorem 2. For any computable functions s{n),m{n) and a{n) with n > s(n) > a(n) + 
71ogn and m{n) < s{n) — 71ogn, there exists a computable ensemble of functions E : 
{0, 1}" X {0, 1}" {0, 1}™(") such that for all x and y in {0, 1}" 

— if C{x) > s(n), C{y \ n) > s(n) and C{y \ n) — C{y \ x) < a{n) 

— then C{E{x,y) \ x) > m{n) — a{n) — 0(1). 

Theorem 3. Let a{n) be a computable function. For every sufficiently large natural number 
n, for every x G {0, l}" such that C{x) > a{n) + 81ogn, 

|R , J > J_ . nn--~a(n) _ 8r,a(n) 

for some positive constant C. 
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Proof. Let m = a{n) + c and s = a{n) + 8 log n, where c is a constant that will be specified 
later. Consider E : {0, 1}" x {0, 1}" — > {0, 1}*" the Kolmogorov extractor given by Theorem[2] 
for these parameters. Let z E {0, 1}*" be the string that has the largest number of E 
preimages in the set {x} x {0, 1}". Note that, for some constant ci, C{z \ x) < ci, because, 
given X, z can be constructed from a table of E, which at its turn can be constructed from 
n which is given because it is the length of x. On the other hand, if y G {0, l}" is a string 
with C{y \ n) > s and C{y \ n) — C{y \ x) < a{n), then Theorem [2] guarantees that, for 
some constant C2, C{E{x,y) \ x) > m — a{n) — C2 = c — C2 > ci, for an appropriate c. 
Therefore all the strings y such that E{x,y) = z are bad for extraction, i.e., they belong to 

{y e {0, 1}" I C{y \n) <s}U{y£ {0, 1}" \C{y\n)>s and C{y \ n) - C{y \ x) > a}. 

Since there are at least 2"~™ such strings y and the first set above has less than 2* elements, 
it follows that 

\{y G {0, 1}" I C{y I n) - C{y \ x) > a{n)}\ > 2" - 2^ = 1 ■ 2"-"(") - n^2''^''\ 

2^ 

This concludes the proof. | 

The proof of Theorem[T]actually shows more: The lower bound applies even to a subset of 
Ax^a containing only strings with high Kolmogorov complexity. More precisely, if we denote 
= {y G {0, 1}" I C{y) > s and C{y) - C{y \ x) > a}, then |^.,„,n-5iogn| > 2^2"-". 
Note that there is an interesting "zone" for the parameter s that is not covered by this result. 
Specifically, it would be interesting to lower bound the size of Ax^a,n- This question remains 
open. Nevertheless, the technique from Theorem [3] can be used to tackle the variant in 
which access to the set i? = {n S {0,1}" | C{u) > \u\} is granted for free. Thus, let 
A^,a,n = {y e {0, 1}" I C^'iy) > n and C«(y) - [ x) > a}. 

Proposition 1. For the same setting of parameters as in Theorem\^ l^xanl > ^•2"-"("), 
for some positive constant C . 

Proof. Omitted from this extended abstract. | 



4 Pairwise independent strings 

We show that if the n-bit strings xi, . . . , are pairwise a- independent, then t < poly(n)2". 
This upper bound is relatively tight, since there are sets with (l/poly(n)) • 2" n-bit strings 
that are pairwise a-independent. 

Theorem 4. For every sufficiently large n and for every natural number a, the following 
holds. If xi, . . . ,xt are n-bit strings that are a-independent, then t < 2n^ ■ 2". 

Proof. There are less than 2""*"^ " strings with Kolmogorov complexity less than a+3 log n. 
We discard such strings from xi, . . . ,xt and assume that xi, . . . ,xt' are the strings that are 
left. Since t < 2"+3iogn _^ ^ ^ ^gg^j gj^^^ ^.j^^t t' < n^2°'. 

For 1 < i < t', let ki = C{xi) and let x* be the shortest description of Xi as described 
in the Preliminaries. Let /3 = a + 31ogn (we assume that a <n — Slogn, as otherwise the 
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statement is trivial). We show that the prefixes of length (3 of the strings xi, . . . , are all 
distinct, from which we conclude that t' < 2^^ = • 2". 

Suppose that there are two strings in the set that have equal prefixes of length /3. W.l.o.g. 
we can assume that they are xi and X2- Then 

C{xl 1 xl) < {ki - /3) + log /3 + 2 log log /? + 0(1), 

because, given x\ can be constructed from /3 and the suffix of length ki — (3 of x\. Note 
that 

C{xl 1 X2) < C{xl I x^) + logA;2 + 21oglogA:2 + 0(l), 

because X2 can be constructed from X2 and k2- Also note that C{xi \ X2) < C{xl \ X2)+0(l). 
Thus, 

C(xi I X2) < C{xl I x^) + logA;2 + 21oglogA:2 + 0(l). 

Therefore, 

C(xi) - C{xi \ X2) >ki- {C{xl I X*) + log A;2 + 2 log log k2 + 0(1)) 

>ki- (ki - /3) - log /3 - 2 log log /? - log A:2 - 2 log log k2 - 0(1) 
> /3 — 3 log n = a, 

which is a contradiction. | 

The next result shows that the upper bound in Theorem |4] is relatively tight. It relies 
on the well-known Turan's Theorem in Graph Theory [H], in the form due to Caro (un- 
published) and Wei [15j (see P, page 248]): Let G be a graph with n vertices and let di be 
the degree of the i-th vertex. Then G contains an independent set of size at least ^ 

Theorem 5. For every natural number n and for every natural number a satisfying 
51ogn < a < n, there exists a constant C and t = ■ 2° n-bit strings xi, . . . ,xt that are 
pairwise a-independent. 

Proof. Let (3 = a — 5 log n. Consider the graph G = {V, E), where V = {0, 1}" and (u, v) £ E 
iff C{u) - C{u \v)>(3 and C{v) - C{v \u)> (3. Note that for every u G {0, 1}", the degree 
of u is bounded by |yl„,^| < 2""^+^ for some constant c. Therefore, by Turan's theorem, 
the graph G contains an independent set / of size at least 2" ■ > 2^~^~^ = • 2". 

For any two elements ti, v in /, we have either C{u) — C{u \ v) < 13 ox C{v) — G{v \ u) < (3. 
In the second case, by symmetry of information, G(u) — C{u \ v) < (3 + 51ogn = a. It 
follows that the strings in / are pairwise a-independent. I 

5 Mutually independent strings 

In this section we show that the size of a mutually a-independent tuple of n-bit strings is 
bounded by poly (n) 2". 

For u G {0, 1}'', we define Do,{u) = {x G {0, 1}" | u G ^x,a} = {x G {0, 1}" | C{u) - 
C{u I x) > a} and da{u) = \Da{u)\. 
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Lemma 1. For every natural number n sufficiently large, for every natural number a, and 
for every u £ {0, !}"■, with C{u) > a + 12 log n, 



2ni2" 

Proof For every x G ^„,a+5iogn, 

C{x) — C{x \ u) > a + 5 log n 
which by symmetry of information implies 

C{u) — C{u \ x) > a + 5 log n — 5 log n = a, 
and therefore, u G ^^.a- Thus 

1 1 

J /'„,\ \ I /I I \ ora-g-Slogn _ pn-g 

UaKU) ^ |^«,a+51og7i| ^ 2?t7 ~ 2n^ 

For every n G {0, 1}", 

^ C(m) - C7(n I x) > a 

=^ C(x) — C(a; I n) > a — 51ogn 

^C{x\u)<n — a + b log n. 

Thus, d„(n) < |{x G {0, 1}" | C(x | u) < n - a + 5 log n}l < • 2""". | 

Since for any string x and natural number a, |Aa;^Q,| < 2"""""^, for some constant c, 
it follows that we need at least T = 2""'^ strings xi, . . . ,xt to "a-cover" the set of n-bit 
strings, in the sense that for each n-bit string y, there exists Xi, i G [T] such that y is 
a-dependent with Xj. The next theorem shows that poly(n)2" strings are enough to a-cover 
the set of n-bit strings. 

Theorem 6. For every natural number n sufficiently large, for every natural number a, 
there exists a set B C {0, 1}" of size poly(n)2" such that each string in {0, 1}" is a- 
dependent with some string in B, i.e., {0, 1}" = UzG-B^^."' -^ore precisely the size of B is 
bounded by {2n^^ + n^^) • 2". 

Proof, (a) We choose T = 2n^^2° strings xi, . . . ,xt, uniformly at random in {0, 1}". The 
probability that a fix u with C (u) > a + 12 log n does not belong to any of the sets A^^^a, for 
z G [T], is at most (1 — 2n^^)'^ < (by Lemma [T|). By the union bound, the probability 
that there exists u G {0, 1}*^ with C{u) > a + 12 log n, that does not belong to any of the 
sets Ax^^a, for i G [T], is bounded by 2" • e~" < 1. Therefore there are strings xi, . . . ,xt in 
{0, 1}" such that IJAx^^a contains all the strings u G {0, 1}" having C{u) > a + 12 log n. 
By adding to xi, . . . ,xt, the strings that have Kolmogorov complexity < a + 12 log n, we 
obtain the set B that a-covers the entire {0, 1}". I 

To estimate the size of a mutually a-independent tuple of strings, we need the following 
lemma. 
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Lemma 2. Let a, 13 € N and let the tuple of n-bit strings (xi, X2, ■ ■ • , a^fc) satisfy 
C{xi . . . Xk) > C{xi) + . . . + C(xfc) — (3. Then there exists a constant d such that 

\A^,^^ n . . . n A^^J < dn^'=+5A;32"-^"+/3. 

Proof. Let u G {0, 1}" be a string in A^^^a H . . . H A^^^a- Then C{u) — C{u \ xi) > a, for all 
i G [k]. Therefore, by symmetry of information, C{xi)—C{xi \ u) > q — 51ogn, for all i € [k]. 
It follows that for every i £ [k], there exists a string pi of length \pi\ < C{xi) — a + 51ogn 
such that, given n, is a descriptor of Xi (i.e., U{pi,u) = Xi). The strings pi, . . . ,pk describe 
the string xiX2 ■ ■ ■ Xk, given u, and therefore 

C{xiX2 ...Xklu) < \pi\ + ... + \pk\ + 21og \pi\ + ... + 21og \pk\ + 0(1) 

< C(xi) + . . . + C{xk) - /ca + 5A: log n + 2 log |pi I + . . . + 2 log \pk\ + 0(1) 

< C{xi) + ... + C{xk) -ka + 7k log n + 0(1) 

< C{xi . . . Xk) + ^ - ka + 7klogn + 0(1). 

So, 

0(xi . . . Xk) - C{xi . . . Xk \ u) > -{/3 - ka + 7klogn + 0(1)). 
By symmetry of information, 

C{u) — C{u \ xi . . . Xk) > C{xi . . . Xk) — C{xi . . .Xk \ u) — 2 logC{u) — 2 log 0(xi . . . Xku) 

— 41oglog 0(xi . . . Xku) — 0(1). 

It follows that 

C{u) — 0(n I xi . . . Xk) > —{P — ka + 7k logn) — Slog n — 3 log k 

and thus 

C{u I xi . . . Xfc) < C{u) + /3 - A;a + (7A; + 5) log n + 3 log k 

< n + (3 - ka + {7k + 5) log n + 31og A; + 0(1). 

Therefore, 

^xi.aH. . .nAa;fe,a ^ G {0, 1}" | C{u \ xi . . . Xfc) < n+/3-A:a+(7A;+5) log n+3 log A;+0(1)}. 

The conclusion follows. | 

Finally, we prove the upper bound for the size of a mutually a-independent tuple of 
n-bit strings. 

Theorem 7. For every sufficiently large natural number n the following holds. Let a be an 
integer such that a > 71ogn + 6. Let (xi, . . . ,xt) be a mutually a-independent tuple of n-bit 
strings. Then t < poly (n) 2". 

Proof. By Theorem [U there exists a set B of size at most poly(n)2"+^i°g" such that every 
n-bit string x is in Ay a+^iogm for some y £ B. We view {xi, . . . , xt} as a multiset. Let 
y be the string in B that achieves the largest size of multiset Ay^a+5\ogn H {xi,...,xt} 
(we take every common element with the multiplicity in {xi, . . . , x^}). Let k be the size of 
the above intersection. Clearly, k > t/\B\. We will show that k = poly(n), and, therefore, 
t<k-\B\ = poly(n) • 2". 
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Without loss of generality suppose Ay^a+5iogn H {xi, • • • , Xt} = {xi, . . . , x^} (as multi- 
sets). Since, for every z G [k], C{xi) — C{xi \ y) > a + 51ogn, by symmetry of information, it 
follows that C{y)—C{y \ Xj) > a. Thus y G A^^^^'^. . .{~^An^^^a■ In particular, Ax^^a'^- ■ -^^x^.a 
is not empty. We want to use Lemma[2]but before we need to estimate the difference between 
C(xi . . . Xk) and C(xi) + . . . + C(xfc). 

Claim. C{xi ...Xk)> C{xi) + . . . + C{xk) - /3, where j3 = a + 41og(nt/2). 
Proof of claim. Suppose C{xi . . . Xk) < C{xi) + . . . + C{xk) — /3. Note that 

C{xi ...xt)< C(xi ...Xk) + C{xk+i . . . Xt) + 2 log C{xi . . . xt) + 0(1) 
< C(xi) + . . . C{xk) + C(xfe+i . . . Xt) - /? + 2 log fen + 0(1). 

Since C{xi . . . xt) > C'(xi) + . . . + C{xt) — a, it follows that 

C{xk+i) + ... + C{xt) -a< Cixk+i ...Xt)- f3 + 2logkn + 0(1). 

On the other hand, 

Cixk+i ...xt)< Cixk+i) + ... + C{xt) + 2 log(t - k)n + 0(1). 

It follows that 

/? - a < 2 log /cn + 2 log(t - k)n + 0(1). 
However, from the definition of /3, 

/3 - a = 41og(nt/2) > 21ogA:n + 2 log(t - /c)n + 0(1). 

The contradiction proves the claim. | 
Now, by Lemma [21 

= drJ'^^^ ^32r!.-(fc-l)a+4 log i+4 log(n/2) 
< (;n7fc+5fc325n-(A:-l)a+41og(n/2)^ 

where in the last line we used the fact that t < 2". 

It can be checked that if a > 7 log n + 6 and k > n, then the above upper bound is less 
than 1, which is a contradiction. It follows that k < n. I 

6 Final remarks 

This paper provides tight bounds (within a polynomial factor) for the size of A^^a (the set 
of n-bit strings that have a-dependency with x) and for the size of sets of n-bit strings that 
are pairwise a-independent. 

The size of a mutually a-independent tuple of n-bit strings is at most poly(n)2". We do 
not know how tight this bound is and leave this issue as an interesting open problem. 

We have recently learned about the paper |5j, which obtains similar results regarding 
the size of sets of pairwise and ^-independence strings, for a notion of independence that is 
suitable for strings with large Kolmogorov complexity. 
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Symmetry of Information Theorem 

Theorem 8. For any two strings x and y, 

(a) C{xy) < C{y) + C{x \ y) + 2 log C{y) + 0(1). 

(h ) C{xy) >C{x) + C{y\x) -2 log C{xy) - 4 log log C{xy) - 0(1) . 

(c) If \x\ = \y\ = n, C{y) - C{y \ x) > C{x) - C{x | y) - 51ogn 

Proof (sketch): (a) is easy and (c) follows immediately from (a) and (b). We prove (b). 
Let C{xy) = t, A = {iu,v) \ C{uv) < t}, Au = {v \ C{uv) < t}. Note that \A\ < 2*+^ Let 
e = [log I^I^^IJ. Let B = {u\ > 2«}. Note that x e B and \B\ < |A|/2^ < 2*-^+^. 

FACT: x can be described by: t, rank in B (which is written on exactly t — e + 1 bits so 
that e can be also reconstructed), 0(1) bits. So C{x) < (t — e + 1) + log t + 21og logt + 0(1). 

FACT: y, given x, can be described by: t, rank in A^, 0(1) bits. So, C{y | x) < e + 
logt + 21oglogt + 0(l). 

Combining the last two: C{x) < t - {C{y \ x) -\ogt - 2 log log t - 0(1)) + logi + 
2 log log t + (1) = C{xy) - C{y \ x) + 21ogt + 4 log log t + 0(1) = C{xy) - C{y \ x) + 
2 log C{xy) + 4 log log C{xy) + 0(1). 
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